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Abstract 



The authors were interested in studying the effects of 
various discourse variables on the psycholinguistic processing 
of sentences within paragraphs by competent adult readers. 
Eye movement measures were used as the dependent variables 
reflecting psycholinguistic processing. When all of the 
constraints within which the study had to be performed were 
examined , it became obvious that none of the traditional 
research designs would be appropriate. Hence , a new experi- 
mental design was conceived v;hich met the constraints of the 
study. This design has some features which make it very use- 
ful for research in many different fields of education. 
Furthermore , from a statistical analysis point of view, this 
design has several desirable characteristics. In this paper , 
a description of the study along with its constraints is 
given. It is then explained v/hy none of the traditional 
designs can be used. The design that was conceived by the 
authors is introduced. The statistical analysis techniques 
to be employed on data collected using this design are then 
explained in detail. 



The authors were interested in studying the effects of 
various discourse variables on the psycholinguistic processing 
of sentences within paragraphs by competent adult readers. 
When all of the constraints within which the study had to be 
performed were examined , it became obvious that none of the 
traditional research designs would be appropriate. Hence, a 
new experimental design was conceived which met the constraints 
of the study. This design has some features which make it 
very useful for research in many different fields of education. 
Furthermore', from a statistical analysis point of view, this 
design has several desirable characteristics. In this paper, 
a description of the study along with its constraints will be 
given. It will then be explained why none of the traditional 
designs can be used. The design that was conceived by the 
authors will then be introduced. The statistical analysis 
techniques to be employed on data collected using this design 
will be explained in .detail. 

Background 

In this study the effects of various discourse variables 
on the processing of target syntactic structures within para- 
graphs was to be explored. The discourse variables of interest 
were aspects of passage coherence; specifically, the grounding 
of the target sentences. Grounding refers to the method in 
which the paragraph is constructed around the target sentence 
and has three levels, foregrounded, backgrounded, and inferred. 



A paragraph is called foregrounded (Symbol = F) if the paragraph 
explicitly introduces , thematizes , and foreshadows the infor- 
mation in the target sentence, A paragraph is called back- 
grounded (Syiruool = B) if it is identical to a foregrounded 
paragraph except that two sentences of semantically neutral 
filler are added immediately prior to the target sentence in 
order to background the concepts in the target sentence. A 
paragraph is called inferred (Symbol =1) if it is coherent 
in nature and no information within the target sentence is 
explicitly introduced prior to the target sentence. The sen- 
tence variables of interest were the syntactic structures of 
the target sentences * specifically , the embeddedness of the 
target sentences. Liobeddedness refers to the r grammatical 
structure of a sentence and has two levels , left and right. 
A sentence is said to be right-embedded (Symbol = R) if its 
grammatical structure is Subject + Verb + Object + Relative 
Clause and left-embedded if its structure is Subject + 
Relative Clause + Verb + Object. 

Thus, when embedding and grounding conditions are crossed, 
a total of six types of paragraphs are obtained. That is, RP 
(a right-embedded structure within a foregrounded paragraph) , 
RB, RI, LP, LB, and LI. The main purpose of the study was to 
determine if there was a difference in the ease of processing 
of the target syntactic structures within these six types of 
paragraphs. Ease of processing was defined by individuals' 
scores on several different variables. These variables were 
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measures of eye movement behavior while reading the target 
sentence and were obtained through the use of an eye movement 
camera. Since it is possible that the subject matter content 
of a paragraph could also affect the visual processing of a 
target sentence/ the subject matter content was varied across 
paragraphs. For reasons which will become clear below, sub- 
ject matter content has six levels (i.e., six different 
subject matter areas were used) . 



One possible design that could have been used was to ob- 
tain some multiple of six individuals and randomly give each 
individual one paragraph to read. Since variance of subject 
matter was also important, in order to gain sufficient power 
for the testing of main effects, many individuals (e.g., 108 
if only three different subject matters were used) would have 
to be involved in the study if each individual read only one 
paragraph. However, because of the time and complexity in- 
volved in obtaining and analyzing visual processing data 
through the use of an eye movement camera, the number of in- 
dividuals used in such a study has to be limited to a man- 
ageable number. For this study the manageable limit was 40. 
Hence, some design has to be used where each individual was 
given more than one paragraph to read. Once repeated measures 
are introduced on the same person, researchers must be careful 
to control for any effects that the order of presentation of 
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the measures may have on the results of the study. This is 
especially important in the area of reading research. Hence, 
it was decided that a variation in the order of the presenta- 
tion of paragraph conditions and of the subject matter content 
must be built into the design. 

Given the constraints just described, the design to be 
introduced presently was conceived. Since there are only six 
embedding x grounding paragraph combinations, it was decided 
that each individual should receive each of these six para- 
graph combinations exactly once. Also, the same subject matter 
content could not be used twice with any of the individuals 
because of fear of memory effects. Hence a different content 
had to be used, within each individual, for each embedding x 
grounding combination. So, it was decided to use six differ- 
ent subject matter areas. Further, in order to avoid the 
complications of unbalanced designs, each subject matter con- 
tent had to appear the same number of times within any par- 
ticular embedding x grounding combination. Latin squares are 
ideal under this set of constraints. 

Consider, for the moment, a study with only six individ- 
uals. Figure 1 illustrates a 36 cell matrix with two dimen- 
sions, individuals and embedding x grounding combinations. In 
this figure the symbols are as defined earlier. A subject 
matter content is then assigned to each cell so that no person 
nor any embedding x grounding combination receives the same 
content twice. There are many ways of completing this assign- 
ment process; Figure 2 illustrates one of these ways. See 
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Combinations 



Combinations 



03 
H 

as 

> 
•H 

H 





LF 


LB 


LI 


RF 


RB 


RI 


1 














2 














3 














4 














5 


i 












6 


1 











CO 
iH 

ttJ 

> 

•H 
H 







LB 


LI 


RF 


FJB 


Rlt 


1 


A 


B 


C 


D 


E 


F 


2 


B 


D 


£ 


C 


F 


A 


3 


F 


C 


A 


£ 


B 


D 


4 


JD 


A 


B 


F 


C 


E 


5 


C 


E 


F 


A 


D 


B 


6 


£ 


F 


D 


B 


A 


C 



Figure 1 



Figure 2 



Denes and Keedwell (1974) for a listing of other possibilities. 
In Figure 2 the letters A, B, C, D, E, and F represent the 
six chosen subject matter content areas. Figure 2 represents 
what is traditionally called a Latin square design. 3ut, this 
design suffers from a lack of power because cf the small number 
of individuals used. To increase the power, the just-illustrated 
process of using a Latin square to assign content x embedding x 
grounding discourse conditions to individuals is repeated as 
many times as feasible, each time using a new sample of six 
individuals. 

But, the order of the embedding x grounding conditions 
still must be varied. One way of doing this is to employ a 
new Latin square which assigns orders to groups of individuals. 
It is this use of Latin squares matched with other Latin 
squares that is the unique feature of this particular design. 
Figure 3 illustrates how this matching is done. The row 
headings refer to groups of six individuals each. Within 
each group, the individuals have already been assigned to 
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content x embedding x grounding discourse conditions by the us€* 
of a Latin square of the form of Figure 2. The order of the 
paragraphs for each group is then determined by reading across 
the rows of Figure 3. For example, for the first group of six 
individuals the order of presentation is the left-embedded 
inferred paragraph first, then the left-embedded foregrounded 
paragraph followed by the right-embedded inferred, the 
left-embedded backgrounded, etc. 1 

Order 



w 

Q* 

o 
u 
o 





f If 2 1 


' 3 


[ 4f 5 


r 6 


1 


LI 


LF 


RI 


LB 


RB 


RF 


2 


LF 


LB 


LI 


RF 


RI 


RB 


3 


LB 


RF 


LF 


RB 


LI 


RI 


•4 


RF 


RB 


LB 


RI 


LF 


LI 


5 
6 


RB 


RI 


RF 


LI 


LB 


LF 


RI 


LI 


RB 


LF 


RF 


LB 



Fiaure 3 



erJc 



Because of this matching of the Latin squares the number 
of individuals included in the study must be some multiple of 
36* This particular study was restricted to exactly 36 indi- 
viduals because, as was explained earlier, it was felt that 72 
individuals would be too unwieldly. Although the matched 
Latin squares design is very similar to a Graeco-Latin square 
type design, it is not the same. In fact, Graeco-Latin squares 
could not have been used for this study because no Graeco-Latin 
square of order 6 exists. In the remainder of this paper the 
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focus will be on the analysis of a matched Latin square design 
involving only 36 individuals. The analysis when a multiple 
of 36 individuals is used is straightforward and will not be 
discussed here. 

Analysis 

The independent variables or factors in this study are 
embedding condition, grounding condition, subject matter 
content, and order of presentation. The matched Latin square 
design allows the researcher to study all of these factors. 
An additional factor is the individuals who are nested wirhin 
order of presentation. For ease of later explanation, suppose 
temporarily that embedding conditions, grounding conditions, 
subject matter content, and individuals are crossed rather 
than prescribed by the six Latin squares of the type displayed 
in Figure 2. That is, suppose that each individual is given 
all 36 of the possible embedding x grounding x content com- 
binations instead of just the six prescribed by the rows of 
the Latin squares of the type in Figure 2. 

If an ANOVA table were set up for this supposed crossed 
design, the columns corresponding to Sources of Variation, 
degrees of freedom, and Expected Mean Squares would be as in 
Table 1. In determining the Expected Mean Squares it was 
decided to treat embedding and grounding conditions as fixed 
and subject matter content and individuals as random. In 
Table 1 the Expected Mean Squares are given when order is 
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considered both as fixed and as random. The authors have 

chosen to consider order of presentation as random because they 

wanted to be able to generalize to orders other than those 

in Figure 3, but we realize that others may want to consider 

2 

order as fixed. 

Recall that a complete factorial design was not really 
used here. Instead, a design based on matched Latin squares 
was used. The use of Latin squares causes several Sources of 
Variation to be confounded. In order to remove this confound- 
ing the researcher must assume that certain Sources of Variation 
are negligible in the setting being studied. For the study 
being described here the assumption was made that all inter- 
actions, except ExG and ExGxO, involving two or more cf E, G, 
S, and 1:0 are considered to be negligible. This assumption 
was chosen both because it made sense from a psycholinguistic 
point of view and because it is the assumption needed to insure 

that a test exists for the main effects of embedding and ground- 

3 

ing conditions, which are of most interest in this study. 
Making this assumption reduces Table 1 to Table 2. 

By examining Table 2 the tests of the various Sources of 
Variation can be derived. These tests are given in Table 3. 
The Mean Squares, except for MS £ es ici ua i' are & H computed 

by the formulas that would be used in the usual ANOVA of the 
crossed design presented in Table 1. To compute MS jj es i^ ua i 

the following expression is used: 
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SS_ - (SS E+ SS G+ SS s+ SS 0+ S Sl :0 +SS ExG +SS ExO +SS GxO +SS SxO +SS ExGxO ) 



'Total 



120 

Notice that both when order of presentation is fixed and when 
order is random that the main effect for Order is not testable. 
A test of order effects, which is needed to be able to deter- 
mine whether fatigue has occurred, is given by looking at the 
design in a slightly different manner: 
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It is realized that the scores will differ widely within each 
time point because different individuals have read different 
paragraphs. But, if no fatigue (i.e., order) effects are 
present it would be expected that the means for all six time 
points would be identical. The A \NOVA table when the design 
is looked at in this manner with individuals considered as 
random and Temporal Order as fixed is 
Sources of Variation 



Temporal Order (T) 
Individuals (I) 
Txl 



df 
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35 

175 



Expected Mean Squares 
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e I 
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a e + a TxI 
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Hence, by inspection of this ANOVA table, it can be seen that 

MS m 
T 

the order effects are tested using F ff 17C . = . 

~> L/ s MS m 

Txl 

Summary 

In this paper a new experimental design, which the authors 
call a matched Latin squares design 9 was introduced. The paper 
provided the motivation for this design and a detailed discussion 
of the methods of data analysis to be used when this design 
is employed. This design has several advantages. Sometimes 
researchers employ traditional well-known designs which allow 
them to test some of their hypotheses of interest, but which 
also force them to ignore other hypotheses they are interested 
in. The first advantage of the matched Latin square design is 
that it allows the researcher to test a larger set of hypothe- 
ses than is possible under other designs employing the same 
number of individuals. For the study which motivated the crea- 
tion of the matched Latin squares design, all of the hypotheses 
of interest become testable once this design is employed. A 
second advantage of the matched Latin squares design is that 
it allows the researcher to gain sufficient power for the 
hypothesis tests while using only a small number of individuals. 
Those readers interested in examining the results of the 
analyses described in this paper are referred to either Pearce 
(1981) or Pearce, Bader, and Blumberg (1983). 
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Footnotes 

Notice that Figure 3 is a special type of Latin square 
which is called complete (or balanced or counterbalanced, 
depending on the source) . A Latin square is said to be com- 
plete if every "treatment" follows each other "treatment" 
exactly once. For this study, the "treatments" are the 
embedding x grounding conditions. 

Actually the choice is between whether order of presen- 
tation is fixed or finite. If one supposes that order is 
finite then the correction factor in the Expected Mean Squares 

would be 1 7 ^o ' which is approximately equal to 1. Hence 

the correction factor can be ignored and order can be treated 
as if it were random rather than finite. 

3 This assumption follows the pattern of assumptions used 
by Winer (1962) in his chapter on Latin squares. Plan 8 in 
that chapter is the plan which is most similar to matched 
Latin squares. 
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TABLE 1(b) 

ANOVA TABLE FOR CROSSED DESIGN (with Order random) 
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Souz'ces of 
Variation df 



Expected Mean Squares 
When Order is Random 



Expected Mean Squares 
When Order is Fixed 



Residual 120 
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TABLE 2 
ANOVA TABLE FOR MATCHED 
LATIN SQUARES DESIGN 
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Source of 
Variation 



F Test When 
Order is Random 



MS. 



F = 



1,5 MS 



ExO 



MS, 



2,10 MS 



GxO 



MS, 



5,25 MS 



SxO 



F Test When 
Order is Fixed 

MS„ 



1,120 MS 



Residual 



MS, 



2,120 MS 



Residual 



MS, 



5,120 



MS 



Residual 



1:0 



ExG 



ExO 



GxO 



SxO 



ExGxO 



Not Testable 
MS 



1:0 



30,120 MS 



Residual 



MS 



ExG 



2,10 



MS. 



ExGxO 



MS 



ExO 



5,120 MS„ . , . 

Residual 

MS^ „ 
GxO 



10,120 MS„ . 
' Residual 

SxO 



25,120 MS„ . , -t 
' Residual 

MS 

ExGxO 



10,120 MS 



Residual 



Not Testable 
MS 



1:0 



30,120 MS 



Residual 



MS 



ExG 



2,120 MS 



Residual 



MS_ ^ 
? ExO 

' 5, 120 s3 MS„ . , - 
9 Residual 

MS, 



GxO 



10,120 MS 



Residual 



MS 



SxO 



25,120 MS 



Residual 



MS 



ExGxO 



10,120 MS 



Residual 



TABLE 3 
HYPOTHESIS TESTS 
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